LOGGING OPPORTUNITIES IN ONLINE PROGRAMS FOR SCIENCE (LOOPS): STUDENT AND TEACHER LEARNING
ANNUAL REPORT 2008
PARTICIPANTS
Many participants contributed to the LOOPS project during the first year of the project. The primary team from Concord Consortium includes:
Robert Tinker, PI, oversees all aspects of the scientific and educational quality of the project.
Paul Horwitz, co-PI, is a Senior Scientist working on curriculum design and reporting strategies
Ken Bell, Project Director, supports the entire project team with planning and scheduling of the various project meetings and collaborations
Stephen Bannasch, Director of Technology, is responsible for the underlying project architecture.
Scott Cytacki, senior software engineer, supervises the development of software for the project
Aaron Unger, junior programmer, works on OTrunk models, Reporting, and the supporting SAIL/OTrunk frameworks.
Others from CC who have participated in the project this year are: junior programmers Sam Fentress and Seong June Kim, curriculum developer Carolyn Staudt, Paul Burney CC webmaster, Ben Greslick CC system administrator and Cynthia McIntyre Director of Communications at CC.
The primary LOOPS leadership team from UC Berkeley includes: Marcia Linn (co-PI Director), Kathy Benemann (Project Manager), Doug Kirkpatrick (Program Coordinator). Graduate students include Kevin McElhaney, Helen Zhang, Phil Daubenmeir, and Jenny Chiu. Technology staff includes Hiroki Terashima, Geeff Kwan, and Tony Perritano. Additional staff includes David Crowell and Jon Breitbart.
University of Toronto Ontario Institute of Science Education participants include: Jim Slotta, co-PI and the Director of the LOOPS project at the University of Toronto. Rokham Sadeghnezhadfard, David Leung and Ali Ajellu technology developers, Cheryl Madeira and Naxin Zhao PhD student researchers.
Partner schools and Teachers
Glenbrook MS, Mt. Diablo USD: Concord CA Mike Marshall
Valley View MS, Mt. Diablo USD: Pleasant Hill, California Matt Hesby
Martinez JHS, Martinez USD: Lauren Norse & Jefferson Hartman
Albany MS, Albany USD: Eric Mapes & Marty Place
Other collaborators or contacts
Consultants - Jeffrey Schoonover Curriculum Developer, Barbara Buckley Project Evaluator
Activities
The goal of the LOOPS project is to use the cyber infrastructure to provide resources that support inquiry in the middle school science classroom. The project will make innovative use of technology to create timely, valid, and actionable reports to teachers by analyzing assessments and logs of student actions generated in the course of using online curriculum materials. These reports will enable teachers to make data-based decisions concerning alternative teaching strategies.
The project is a collaboration of the Concord Consortium with a research group at the University of California, Berkeley, and another group located at the University of Toronto. In order to coordinate the efforts of these remotely-situated teams, during the first year of the project we organized six face-to-face meetings, at Concord and at the other two locations, and we have held conference calls every two weeks. Below, we report on the activities of each participating team separately.
List dates and locations of face\-to\-face meetings.
Activities at the Concord Consortium.
In addition to its responsibility for the overall management of the project, the Concord Consortium's main role has been to design and implement the infrastructure technology required to make it a success. In accomplishing this goal, we have adapted and enhanced the SAIL (Scalable Archictecture for Interactive Learning) ref technology developed by the TELS Center on a previous NSF grant give NSF project number and dates of performance. NSF GRANT # ESI0334199 October 2003 to August 2008 Paul Horwitz PI
The Concord Consortium has also played an important role in curriculum development, based on the California 8th grade science standards for the two curricular strands of the project: Force and Motion, and Chemical Reactions.
Development of Technological Infrastructure
Installation.
What have we done to make installation easier for schools? (If nothing, leave this out.)
Curriculum tools
Describe the development of Smart Graphs, Smart Tables, and Smart Models and their use within LOOPS. Any other curriculum tools that LOOPS can justifiably claim credit for?
Authoring
Goal: to enable curriculum developers who are not programmers to author LOOPS activities. Describe the authoring system in its current state, and how it is supported by OTrunk, SDS, and whatever other infrastructure technology. Give a specific example of an authored activity (e.g., the motion probe one) to point out how easy it is to author these.
Reporting
Describe and give examples of LOOPS reports, emphasizing their advantages over previous versions, including tracking teacher-initiated changes, real-time reporting of student work, grouping of students, and support for performance assessment rubrics.
Curriculum development
LOOPS Teacher Dashboard and Information Management
A central feature of LOOPS is the persistence of student data and the use of such data by the classroom teacher. Describe the technologies we are creating so that data generated as students use our technology can be used creatively by teachers both during and after class. Describe the new features of the LOOPS Teacher Dashboard (e.g., managing groups of students) that the SAIL portal doesn't have, thus justifying our development of a second portal. Provide brief description of the process of portal development in JRuby and the advantages of JRuby over Java for that purpose.
LOOPS Portal: Project Service Layer added to TELS Portal
Following the California science standards for the 8th grade, we started by describing 10 modules, covering:
1. Vector position
2. Position-time graphs
3. Velocity-time graphs
4. Velocity in one and two dimensions
5. Forces
6. Separation of forces
7. Force and motion in one dimension
8. Force and motion in two dimensions
9. Gravity
10. Projects
For each of these modules, we identified the standards that the module addressed, the software or other technology that would be required to run it, the nature and purpose of the classroom discussions that support it, the investigations students would be expected to undertake, the extensions to the module that could be introduced, time permitting, suggested lab activities, and how student learning was to be assessed.
With these modules in hand, and advances made on the technologies necessary to support them, we have more recently been engaged in creating teaching and lesson plans that cover the first 3 weeks of an expected 6-week curriculum in force and motion. These materials cover position-time graphs in one dimension, velocity-time graphs in one dimension, and motion in two dimension, all treated from a purely kinematic point of view - in other words strictly as descriptions of motion with notions of causation (e.g, forces) ignored. The second three-weeks, not yet completed, will deal with forces.
In all of our curriculum development we are emphasizing particularly the "loops" - i.e., the feedback and reporting mechanisms - that we intend to build in. Thus, for example, in introducing position-time or velocity-time graphs to students we intend to make use of our Smart Graph technology, which can automatically identify such features as inflection points, monotonic regions, maxima and minima, and so forth, on a graph, whether it is produced by real-world or model data, or drawn by a student. This enables us to comment on the students' work, scaffold their efforts, and report on their success or failure at specific tasks.
University of California, Berkeley activities
The UC Berkeley sub award for the LOOPS project was executed in 2008. Leadership includes Marcia Linn (Director) and Kathy Benemann (Manager), Doug Kirkpatrick (Program Coordinator). Graduate students include Kevin McElhaney, Helen Zhang, Phil Daubenmeir, and Jenny Chiu. Technology staff includes Hiroki Terashima, Geeff Kwan, and Tony Perritano. Staff includes David Crowell and Jon Breitbart. The Berkeley Institutional Review Board has accepted UC Berkeley's LOOPS protocol.
UC Berkeley participants have contributed to a face-to-face meeting in Concord and regular technology and leaders meetings (averaging three times a month). In addition, Bell and Benemann have coordinated on a weekly basis. In addition, we have contributed to the WIKI for LOOPS.
During the first year we have reviewed and pilot tested possible items for baseline assessments, recruited teachers and schools, began the process of defining LOOPS scenarios, and conducted meetings with potential LOOPS users. In addition, we have reviewed possible curriculum and technology designs and considered ways to incorporate LOOPS.
Assessment
Baseline assessments for Force and Motion and Chemical Reactions items were administered to both TELS and non-TELS students based on convenience. The two groups are not comparable as they come from schools with different Academic Performance Index scores. These are all items that have been released by TIMSS or other testing programs.
The pilot tests can be used to establish goals for the new curriculum materials. As the bar chart below shows, performance on Force and Motion items suggests a need for intervention in student learning in this topic area (Figure 1). For schools that did not study TELS, this can be seen as a baseline. As is apparent, students have limited understanding of the topics in these assessments.
INSERT FIGURE 1 here!Berk_figure1!
Figure One: Five items administered to students in TELS schools who either did or did not study any TELS units the previous year.
The Chemical Reactions items were administered to students in similar schools. An analysis of Chemical Reactions items indicates significant advantage to students who experienced TELS instruction consistent with the impact of the unit in the past (Figure 2).
INSERT FIGURE 2 here!berk_fig2!
Figure Two: Seven items administered to students of two TELS teachers.
LOOPS Schools
UC Berkeley has recruited three schools to participate in the LOOPS project and one school to participate as a potential back-up school. All of these schools have participated in past TELS-related research programs and have garnered administrative and district-level support for integrating effective use of technology into science curriculum. Several formal and informal meetings have been organized since June 2008, during which we collaborated with teachers and researchers in the curriculum and technology design process.
We secured pacing guides for physical science courses. We have reviewed these guides as well as the California standards. Since this is the first year of use of the guides we envision some revision after the plan is tried out. The timing of the units and the time devoted to topics appears a bit disjointed. To make LOOPS successful we will need to align the guides with our curriculum design plan.
Our first formal meeting in July provided us with invaluable insight into teacher's successes and challenges when teaching Force and Motion topics to middle school students. Most of the teachers expressed excitement about their prior use of motion probes and indicated interest in continuing to use this technology. The teachers were somewhat confused about the goals of LOOPS and how the program would be enacted in their classrooms. In response we began the process of creating scenarios.
During our second formal meeting in August, attended also by Concord Consortium project manager Ken Bell, we were able to gather teacher perspectives into revising current TELS modules to fit LOOPS research and instructional goals, while also evaluating new technology tools. Kevin McElhaney, who has been serving as the LOOPS graduate student researcher, played an essential role during these planning meetings, providing both teachers and senior researchers avenues for designing activities and the accompanying technology components.
These meetings underscored the importance of clarifying LOOPS scenarios and developing proof of concept technologies. These materials will be discussed with teachers as they become available.
LOOPS Scenarios
We have participated in the iterative process of drafting LOOPS scenarios. These discussions are helping us identify the kinds of feedback systems that might be desirable in the classroom setting. UC Berkeley, Concord Consortium, and Toronto have revised these scenarios. We will soon discuss these ideas with teachers and further refine the scenarios.
We have discussed various curriculum approaches, recently meeting with Tinker at Berkeley to refine the context of instruction and consider appropriate pacing arrangements.
University of Toronto activities
Continued development of the Scalable Architecture for Interactive Learning (SAIL).
Since 2003, Professor Slotta has led an international team of researchers and technology designers in developing SAIL, which is a java-based framework for the development of interactive, interoperable learning materials and environments. SAIL has been the basis of our development activity in the NSF-funded TELS center, and is one of the primary deliverables of that effort. It has served as the basis of all technology development in subsequent projects, including LOOPS
Organized a SAIL technology architecture workshop. June 23, 2008. Utrecht, The Netherlands
Professor Slotta organized, with collaborator Turadg Aleahmad and Stephen Bannasch, a pre-conference workshop where members of the international community were exposed to the core LOOPS technologies and encouraged to design new applications that would connect to their own research. 15 participants gathered in Utrecht, the Netherlands, including representatives from a large European Union Framework 7 project called SCY that is interested in collaborating in technology development, as well as members of prestigious U.S. and Scandinavian labs. This workshop was a full day event, with participants first exploring SAIL and LOOPS technologies, then breaking into focus groups (one on technology architectures and repositories, and another on curriculum and assessments).
Convened a SAIL repository development workshop. August 1-5, 2008. Berkeley, California.
Building on the earlier technology workshop, Professor Slotta invited the two lead technology developers from the SCY (Science Created by You) project to participate in a hands-on development workshop held in Berkeley, California. SCY is a large collaboration project funded by the European Union's Framework 7 program (8.5 Million Euros from 2008-2013). Professor Slotta is a partner in this project, as he is eligible being from a Canadian institution. Slotta is a primary member of the SCY technology architecture and pedagogical agents work packages. To ensure that SAIL is of direct relevance to SCY, Slotta convened a workshop where the two lead programmers from SCY joined the three lead programmers from the TELS center and two programmers from his group at University of Toronto, to develop a new Repository Of Open source Learning Objects (ROOLO) that would interconnect with the existing SAIL portal that all three groups were using. This repository was successfully developed and is now being used in all three locations, with Slotta's team in Toronto taking a lead role.
LOOPS technology development meeting. August 5-8, 2008. Berkeley, California
In conjunction with the fifth (and final) annual TELS retreat, Slotta led a break-out session of the various members of the TELS and LOOPS technology teams to discuss issues and agendas for technology development (for which Slotta has overall responsibility). Several major topics were discussed, including authoring, reporting, portals, and repositories. A SAIL technology retreat was planned where these topics would be more fully discussed, to be held in mid October, in Ontario, Canada.
Ongoing leadership and research meetings.
Professor Slotta has joined regular meetings of the LOOPS leaders where research and technology development is planned. He also convened a weekly technology development meeting, where other members of his technology group participated. Finally, Slotta and two PhD students, Cheryl Madeira and Naxin Zhao, attended bi-monthly meetings of the LOOPS research community.
PUBLICATIONS & PRODUCTS
University of Toronto Publications
Krajcik, J., Slotta, J.D., McNeil, K. and Reiser, B. (2008). Designing Learning Environments to Support Students' Students Constructing Integrated Understanding. In Y. Kali, M. C. Linn, & J. E. Roseman (Eds.), Designing Coherent Science Education. New York: Teachers College Press.
Slotta, J.D. and Peters, V. (2008). A Blended Model for Knowledge Communities: Embedding Scaffolded Inquiry. Proceedings of the International Conference of the Learning Sciences. Utrecht.
Peters, V. and Slotta, J.D. (2008). Building Wiki-Based Pedagogical Scripts for Knowledge Communities. Proceedings of the International Conference of the Learning Sciences. Utrecht.
Slotta, J. D. (in press). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. To appear in Mäkitalo-Siegl, K., Kaplan, F., Zottmann, J. & Fischer, F. (Eds.). Classroom of the Future. Orchestrating collaborative spaces. Rotterdam: Sense.
Slotta, J. D. & Jorde, D. (in press). Learning from our peers in international exchanges: When is worth doing, and how can we help it succeed? The International Journal of Science Education.
Workshops
Slotta, J.D., Aleahmad, T., and Bannasch, S. (2008). The Scalable Architecture for Interactive Learning (SAIL) - New tools and communities for research. Workshop presentation at the International Conference of the Learning Sciences (ICLS). Utrecht, The Netherlands
Conference Presentations:
Najafi, H., Slotta, J. D. & Gelb, D. (2008, August). Promoting knowledge building in graduate-level education: Opportunities and pitfalls . Paper presented at the IKIT Summer Institute, Toronto, ON.
Peters, V., & Slotta, J. D. (2008). Building wiki-based pedagogical scripts for knowledge communities. Proceedings of the International Conference of the Learning Sciences. June 23-27. Utrecht, The Netherlands.
Slotta, J.D. and Peters, V. (2008). A Blended Model for Knowledge Communities: Embedding Scaffolded Inquiry. Paper presented at the bi-annual meeting of the International Conference of the Learning Sciences (ICLS). June 23-27. Utrecht, The Netherlands.
Peters, V., & Slotta, J.D. (2008). Co-designing wiki-based scripts for secondary school biology. Paper presented at the annual meeting of the Canadian Society for Studies in Education. May 31 - June 3. Vancouver, BC.
Najafi, H., Slotta, J., Gelb, D. (2008). Understanding the relationships between curriculum planning decisions, patterns of activity, and learning outcomes: An analysis of the evolution of a course community. Paper presented at the annual meeting of the Canadian Society for the Study of Education. May 31 - June 3. Vancouver, BC.
Peters, V., & Slotta, J. D. (2008). Connecting Knowledge Building With Scripted Activities in a Secondary School Biology Classroom: A Case Study. Paper presented at the annual meeting of the American Educational Research Association, New York, NY.
Concord Consortium Publications
Bannasch, S., & Tinker, R. (2008). A Brief Description of the SAIL Environment (Monograph). Concord, MA: The Concord Consortium.
Levy, D., & Tinker, R. (2008). Links between dynamic representations of atomic-scale phenomena and molecular reasoning Paper presented at the Chais Conference on Instructional Technologies Research.
Tinker, R. (2007). How do students learn from models? Case studies in guided inquiry. @Concord, 11(1), 14-15.
Tinker, R. (2007). How technology will revolutionize STEM learning. Paper presented at the NESDEC Annual Meeting.
Tinker, R. (2007). Learning and Technology: Expanding Opportunities. Paper presented at the National Value of Science Education Conference.
Tinker, R. (2007). Potholes in the road to proving technology. @Concord, 11(1), 2-5.
Tinker, R. (2007). The Educational Accelerator: A virtual facility for educational R&D. Paper presented at the Enabling science discoveries through visual exploration.
Tinker, R. (2008). Exploring the Science of Atoms and Molecules Using the Molecular Workbench. Paper presented at the AAPT Annual Winter Meeting.
Tinker, R. (2008). Learning What Students Are Thinking Paper presented at the AAPT Winter Meeting.
Tinker, R. (2008). Perspective: The Concord Consorium vision. @Concord, 12(1), 2-3.
Tinker, R., & Bell, K. (2008). The Center for Enhanced Learning of Science. @Concord, 12(1), 10-11.
Tinker, R., & Staudt, C. (2008). The Molecular Workbench: Linking the Micro and Macro. Paper presented at the NSTA National Conference.
Tinker, R., & Staudt, C. (2008). The Science of Atoms and Molecules. Paper presented at the NSTA National Conference.
Tinker, R., Staudt, C., & Hazzard, E. (2008). Probes and Models in the High School. Paper presented at the NSTA National Conference.
Tinker, R., & Xie, Q. (2008). Applying computational science to education: the Molecular Workbench paradigm. Computing in Science and Engineering, 10(5), 24-27.
Tinker, R., & Xie, Q. (Invited for 2009). Infusing Science into Science Education. Science.
Tinker, R., Xie, Q., O'Brian, E., & Pallant, A. (2007). Chemistry visualization and student learning: Light-matter interactions using Molecular Workbench models. Paper presented at the AERA Annual Meeting.
Zucker, A., Galvis, A. H., & Tinker, R. (2007). Probeware and the XO. @Concord, 11(1), 1, 4-5.
Berkeley Publications Workshops and Presentations
PublicationsGerard, L. F., Bowyer, J. B. & Linn, M. C. (2008) Principal Leadership for Technology-Enhanced Science. Journal of Science Education and Technology, 17(1), 1-18.
Gerard, L., Bowyer, J., & Linn, M. C. (2008). Scaling Technology-Enhanced Science Curriculum: Leadership Development in a Community of Principals. In International Perspectives in the Learning Sciences: Cre8ting a Learning World. Proceedings of the 8th International Conference of the Learning Sciences (Vol. 3, pp. 35-36). Utrecht, The Netherlands: International Society of the Learning Sciences, Inc.
Liu, O. L., Lee, H.-S., Hofstetter, C., & Linn, M. C. (2008). Assessing Knowledge Integration in Science: Construct, Measures and Evidence. Educational Assessment, 13(1), 33-55.Spitulnik, M. W. & Linn, M. C. (pending review). Professional Development and Teachers' Curriculum Customizations: Supporting Science in Diverse Middle Schools.
Varma, K., Husic, F., & Linn, M. (2008). Targeted support for using technology-enhanced science inquiry modules. Journal of Science Education and Technology, 17(4), 341-356.
Presentations
International Opportunities for Technology-Enhanced Learning
Center for Information Technology in Education (CITE) 10th Anniversary Seminar
University of Hong Kong
May 14, 2008
Technology and Science Learning
Guangxi Normal University
Guilin, Guangxi, China
May 5, 2008
Keynote: Using Dynamic Visualizations and Online Guidance to Bring Learning to Life
Hong Kong IT in Education Symposium 2008, 21st Century Learning @ Hong Kong Conference
King George V Campus, Hong Kong
May 3, 2008
Interactive Visualization and Simulation Tools - Do They Make a Difference?
Wallenberg Global Learning Network
Lund University, Sweden
April 1, 2008
Examining the Role of Teacher Partnerships in Science Education Research, Professional Development, and Teacher Learning (with Keisha Varma & Fred Husic)
National Association of Research in Science Teaching (NARST) Annual Meeting
Baltimore, MD
March 31, 2008
Keynote Address: Science, Technology and Policy
National Association of Research in Science Teaching (NARST) Annual Meeting
Baltimore, MD
March 31, 2008
Add Barbara Buckley material ?
